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ABSTRACT 



This study examined the reliability and validity of a 
group -assessment interview procedure designed to evaluate the verbal, 
interpersonal, and leadership qualities of students applying to a teacher 
education program. The group assessment procedure involved a 90-minute 
session with eight students in which three dimensions of teacher behavior 
were evaluated (verbal abilities, interpersonal skills, and leadership 
qualities) . Data were gathered on 68 student teachers who had previously 
participated in the group assessment process. Researchers also gathered data 
from cooperating teachers' and university supervisors' evaluations of the 
student teachers and students' grade point averages and ACT scores. The 
researchers examined whether: (1) the group assessment process predicted 

future student teaching performance; (2) the group assessment scores were 
reliable across raters; and (3) the group assessment interview was a better 
predictor of student teaching performance than academic criteria. Data 
analysis indicated that the group assessment overall rating predicted student 
teaching performance, and it did so better than academic criteria. The 
results also showed high interrater reliability on the group assessment 
measures. (Contains 31 references.) (SM) 
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Abstract 

This study presents findings on the reliability and validity of a group-assessment interview 
procedure designed to evaluate the verbal, interpersonal, and leadership qualities of students 
applying to a teacher-education program. We examine whether: (a) the group-assessment process 
predicts future student-teaching performance, (b) the group-assessment scores are reliable across 
raters, and (c) the group-assessment interview is a better predictor of student-teaching 
performance than academic criteria. After gathering data from 65 student teachers who had 
previously participated in the group-assessment process, we found that the group-assessment 
overall rating predicts student-teaching performance and does so better than academic criteria. 
Finally, we found high interrater reliability on the group-assessment measures. 
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Evaluating the Use of Group Interviews to 
Select Students into Teacher-Education Programs 
Most teacher-education programs select students based on academic criteria such as grade 
point average (GPA) and standardized test scores (Haberman & Post, 1996). This is curious 
because research findings repeatedly demonstrate that these academic criteria are generally poor 
predictors of who will be successful teachers (Baskin, Ross, & Smith, 1996). Much of the 
research on effective classroom teaching emphasizes the importance of verbal, interpersonal, and 
leadership skills (Shechtman, 1992a; Westbrook, 1998). Despite this body of research findings, 
teacher-education programs often ignore impressive verbal, interpersonal, and leadership 
qualities in student applicants in favor of the pursuit of academically outstanding students. 

This paper examines a group-assessment interview procedure designed to evaluate the 
verbal, interpersonal, and leadership skills of students applying to teacher-education programs. 
Findings fi-om extensive previous research in Israel suggest that the group-assessment procedure 
has strong potential as a process to select promising classroom teachers (Shechtman, 1992a). 
First, after describing the group-assessment procedure, we examine whether group-assessment 
scores are reliable across raters. Second, we investigate whether the group-assessment procedure 
predicts future student-teaching performance. Third, our analyses explore whether the group- 
assessment scores are a better predictor of future student-teaching success than academic criteria 
such as GPA and standardized test scores. We are also interested in whether the success of the 
group-assessment procedure in Israel is applicable to teacher-education programs in the United 




States. 
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Theoretical, Empirical, and Policy Issues 

Effective classroom teaching is associated with a number of skills or qualities possessed 
by successful teachers. Researchers have identified three dimensions of teaching that are related 
to effectiveness in the classroom (Shechtman, 1992a). They are; verbal, interpersonal, and 
leadership skills. 

Effective communication with students is a central facet of successful teaching 
(Shechtman, 1992a). A teacher must possess the verbal skills to think clearly, organize his or her 
thoughts, and express himself or herself with clarity. 

Skill in interpersonal relationships is a crucial trait for successful teachers. Effective 
teachers must be able to consider the social-emotional needs of their students and the affective 
dimensions of learning (Steele, 1999). Successful teachers build support, rapport, and trust with 
their students. 

Third, an effective teacher must have leadership qualities to motivate students through 
self-assuredness, dynamism, and enthusiasm (Westbrook, 1998). Highly successful individuals, 
both minority and nonminority former students alike, reflect back that their most highly 
respected and memorable teachers were those with strong leadership and interpersonal skills 
(Johnson & Prom-Jackson, 1986; Steele, 1999). 

Most teacher-education programs do not consider verbal, interpersonal, and leadership 
skills when selecting students (Goodlad, 1990; Haberman, 1987; Russell, Persing, Dunn, & 
Rankin, 1990). Instead, teacher-education programs typically use academic criteria such as grade 
point average and standardized test scores to select students, even though the research 
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demonstrates that academic criteria are poor predictors of success in teaching (Guyton & 

Farokhi, 1987; Haberman, 1987; Shechtman & Godfned, 1993). 

Baskin, Ross, and Smith (1996) suggest that teacher-education programs persist in using 
academic criteria because of a series of reports during the 1980s that called for teachers 
achieving mastery of subject content (Carnegie Task Force on Teaching as a Profession, 1986; 
Holmes Group, 1986; National Commission for Excellence in Teacher Education, 1985; National 
Consortium for Educational Excellence, 1985). These reports advocated using standardized tests 
to assess the fit between a potential teacher and the objectives of the school district. Still others 
suggest that academic criteria are so extensively employed because there is a perception that 
there is a parallel between what might predict whether a student will be successful at college 
generally and whether a student will be successful in a teacher-education program more 
specifically (Shechtman & Sansbury, 1989). 

Educational researchers have had a difficult time developing appropriate, valid, and cost- 
effective measures to predict success at teaching. Researchers using psychological tests to 
measure the affective characteristics of teaching applicants have reported mixed results (Chang, 
1994; Henjun, 1983; Verdini, 1990). More recently, there has been considerable interest in using 
individual interviews to select students into teacher-education programs. Although several 
researchers have presented evidence that ratings on individual interviews can be good predictors 
of future teaching success (Coleman, 1987; Haberman, 1987; Malvern, 1991), these interviews 
require a tremendous time commitment, with only a few students eliminated from the applicant 
pool (Benner, George, & Cagle, 1987). 




6 



Group Interviews 6 

Another method for selecting students into teacher-education programs that has shown 
some success is the assessment-center model (Gerlach & Millward, 1989). In this procedure, 
students are evaluated on a series of activities over several days to assess their likely success at 
future teaching. A major drawback with the assessment-center approach is the complexity and 
high cost of implementing the typical two- or three-day routine. After implementing an 
assessment-center model at his university, Millward reports (personal communication, 

September 9, 1997) that the procedure was suspended because of the size of the teacher- 
education program and the cost in terms of faculty time. 

Shechtman (1992a) developed a modified version of the assessment-center model that 
addresses the limitations spelled out above. Shechtman reduced the two- or three-day procedure 
to a 90-minute session. Her studies in Israel indicate that the modified procedure that she terms 
Group Assessment is a valid prediction process for future teaching performance (Shechtman, 
1992a). Moreover, it is as effective as two- or three-day routines and it is more effective at 
predicting future teaching performance than academic criteria such as GPA and standardized test 
scores. While Shechtman’s findings indicate success with the Group- Assessment technique in 
educational programs in Israel, there are no systematic studies on group-assessment procedures 
used with teacher-education students in the United States. 

Group-Assessment Procedure 

The Group-Assessment procedure is a 90-minute session involving two trained assessors 
and eight students seeking admission to teacher education. During the 90-minute session, 
students introduce themselves and are encouraged to speak freely, giving their impressions of 
who they are. The group is then given two different controversial topics to discuss followed by a 
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leaderless group activity that involves problem solving. Each group member then provides 
feedback to others in the group. Lastly, members are asked if they feel they have accurately 
reflected who they are during the course of the group’s interactions. While the activities take 
place, assessors evaluate each individual on his or her verbal ability, interpersonal skills, and 
leadership qualities. Each candidate is then assigned an overall rating that portrays his or her 
general fitness for teaching. This last rating is not an average; it may include qualities (e.g., 
emotional stability, integrity, self-awareness, flexibility) not included in the other three (i.e., 
verbal, interpersonal, leadership) categories. An overall rating can be generated by assessors in 
the group-assessment session and can be valid predictors of success, in this case of future 
teaching performance (Thornton & Byham, 1982; Zedeck, 1986). 

Method 

Sample 

The subjects in this study consisted of 68 students who went through the group- 
assessment process; were selected to enter the elementary-education, teacher-education program 
at Utah State University; and, approximately two years later, completed their student teaching. 
Eleven different university instructors participated as evaluators in the series of group- 
assessment interviews. Each group-assessment session consisted of eight students and two 
trained university instructors who served as assessors. 

Procedure 

The group-assessment procedure was a 90-minute session in which three dimensions of 
teaching behavior were evaluated: verbal abilities, interpersonal skills, and leadership qualities. 
In addition, an overall score was obtained for each student. During the 90-minute session, each 
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evaluator independently rated each student on the dimensions of teaching performance (i.e., 
verbal, interpersonal, and leadership) as well as on the overall impression. Following the group- 
assessment session, the evaluators met and arrived at a consensus score for each dimension for 
each student. For a more detailed discussion of the group-assessment procedure, see Shechtman’s 
(1994) manual, A Group Assessment Procedure for Teacher Education Candidate Selection. 
Dependent Measures 

The dependent variables for this study are student-teaching evaluation score given by the 
cooperating teacher and student-teaching evaluation score given by the university supervisor. 
Each student teacher receives a teaching-evaluation score from both the cooperating teacher with 
whom he or she teaches and from a university faculty supervisor. The scores range from 0 to 4, 
with 0 being a poor evaluation score and 4 being an outstanding evaluation score. The mean 
score for cooperating teacher evaluation was 3.47 (S.D .= .47 ). The mean score for university 
supervisor evaluation was 3.32 (S.D. = .54). 

Independent Variables 

Academic criteria. We employed two measures of academic performance, grade point 
average (GPA) and ACT test score. Grade point average was the GPA the student earned in a 
select group of required general education courses before being admitted to the teacher-education 
program. GPA was measured on a scale of 0-4, with 0 being the poorest score and 4 being a 
superior score. The mean GPA was 3.50 (S.D. = .28). ACT test score was the composite score 
earned by a student on the ACT, a standardized test used as an admission criterion at many 
colleges and universities. Scores ranged from 19 to 32; the higher the score the more likely the 
student was to be successful at college studies. The mean ACT score was 23.64 ( S.D. = 2.9). 
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Group-assessment dimensions. There were four dimensions that were evaluated in the 



group-assessment procedure; verbal abilities, interpersonal skills, leadership qualities, and an 
overall rating. Each of these dimensions were measured on a scale of 1-6, with 1 being the 
poorest score and 6 being the most outstanding. For each dimension (i.e., verbal, interpersonal, 
leadership, overall), a consensus score was determined for each student. A consensus score for 
each student was arrived at by the two evaluators at the group-assessment session. The mean 
score for verbal abilities was 4.03 (S.D. = 1.1); for interpersonal skills, the mean was 4.26 (S.D. = 
1.0); and for leadership qualities, the mean was 3.97 (S.D. = 1.1). The mean score for the overall 
rating was 4.28 (S.D. = .98). 

Analyses 

To determine whether the group-assessment scores were reliable across evaluators, we 
used a correlational analysis. For each of the group-assessment dimensions, we correlated the 
scores given by one evaluator with the scores given by the second evaluator. 

To evaluate the relative importance of group-assessment measures versus academic 
criteria for predicting success at student teaching, we estimated regression equations with 
student-teaching scores as the dependent variables and group-assessment measures and academic 
criteria as the independent variables. 

Results 

Interrater Reliability 

Interrater reliability was high for all group-assessment categories. Correlation coefficients 
for pairs of assessors were: (a) verbal abilities, r=.509; (b) interpersonal skills, r=.487; (c) 
leadership qualities, r=.608; and (d) overall rating, r=.663 ( n=57). The p-value for each of these 
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coefficients was less than .0001 . We report the p-value here to give an impression of the 
magnitude of the relationships. Because our sample was not randomly selected, it is 
inappropriate to use tests of statistical significance (Cahan, 2000; Levin & Robinson, 2000). 
However, the coefficients we present here are comparable to results reported elsewhere (see 
Shechtman, 1992b). 

Academic Criteria. Group-Assessment 
Measures, and Student-Teaching Evaluations 

Multiple-regression equations were estimated to determine the relative importance of 
academic criteria versus group-assessment measures to predict successful student-teaching 
performance. The dependent variables in these analyses are student-teaching evaluation scores 
submitted by the student’s cooperating teacher and by the student’s university faculty supervisor. 

Bivariate relationships . We initially generated a zero-order correlation matrix to examine 
the intercorrelations among the variables used in the analyses. The correlational analysis revealed 
at least two noteworthy findings. First, the overall rating from the group-assessment measures is 
highly correlated with the other three group-assessment measures (i.e., verbal [r=.82], 
interpersonal [r=.75], and leadership [r=.74]). This degree of multicollinearity indicates that the 
overall rating and the other three group-assessment measures should not be entered in the same 
regression equation. 

The second noteworthy finding is that the intercorrelation between student-teaching 
evaluation scores for cooperating teachers and university supervisors is high (r=.65). This 
indicates a high degree of interrater reliability between cooperating teachers and university 
supervisors who evaluate student teachers. 
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Multiple-regression findings. Table 1 presents the results of the multiple-regression 
analyses. The group-assessment overall-rating score is the only measure that predicts successful 
student-teaching evaluation scores given by cooperating teachers. The higher the overall-rating 
score, the more likely a student is to earn a high student-teaching evaluation score from the 
cooperating teacher. Overall-rating score accounts for 12% of the variance in student-teaching 
evaluation score. The academic criteria variables, GPA and ACT score, are not associated with 
student-teaching evaluation score from the cooperating teacher. 



Table 1 about here 



Both ACT score and overall-rating score predict successful student-teaching evaluation 
scores given by the university supervisor. The overall-rating score is particularly important. The 
value jumps from .1 1 to .22 when the overall-rating score is entered into the regression 
equation. Interestingly, ACT score is negatively associated with student-teaching evaluation 
score. The lower one’s ACT score, the more likely a student is to earn a high evaluation score for 
teaching from his or her university supervisor. GPA is not associated with student-teaching 
evaluation score given by the university supervisor. Finally, the overall-rating score is a more 
powerful predictor of student-teaching evaluation score than the other three group-assessment 
measures (i.e., verbal, interpersonal, and leadership). 

Further analysis. We encountered a surprising finding. Not only did higher ACT scores 
not predict success at student teaching for the respondents, the relationship between ACT scores 
and student-teaching success was negative. At this point we extended our analysis to identify 
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more precisely the relationships among ACT scores, the overall group-assessment measure, and 
success at student teaching. The evidence on success at student teaching suggests that the trend 
among respondents with the lowest ACT scores is most dramatic as overall group-assessment 
scores increase. Table 2 shows an improvement in student-teaching evaluation scores from 3.2 to 
3.37 to 3.57 as one moves from low to medium to high on the overall group assessment score 
among respondents with low ACT scores. The trend is much less pronounced among respondents 
who had high ACT scores. Their student-teaching evaluation scores changed from 2.87 to 2.67 to 
3.36 as one moves from low to medium to high on the overall group assessment score. 



Insert Table 2 about here 



Discussion 

This study addressed three research questions. First, are group-assessment scores reliable 
across raters? Our results show high interrater reliability. Interrater reliability was highest for the 
overall-rating score. This finding is consistent with the work of other researchers who note that it 
is more difficult to evaluate a student teacher on specific traits than on a global rating of potential 
(Klimoski & Brickner, 1987; Shechtman, 1992b; Thornton & Byham, 1982). Shechtman and 
Sansbury (1989) argue that because the overall rating is the best predictor of student-teaching 
success among the group-assessment measures, and because the overall rating has the highest 
interrater reliability, then the overall rating should be the group-assessment measure that is used 
in the teacher-selection process. 
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The second research question was whether the group-assessment procedure could be used 
to predict future student-teacher performance. The overall rating was associated with student- 
teaching evaluation scores from both cooperating teachers and university supervisors. 

The group-assessment technique has advantages over other instruments and procedures 
that purportedly predict student-teacher success. The group-assessment procedure involves a 90- 
minute session during which eight student-teacher candidates are assessed. The 90-minute 
session is more efficient than two-to-three day assessment-center procedures. The group- 
assessment process is more efficient than individual interviews of candidates, which can be quite 
costly in terms of faculty time and energy without appreciably reducing the pool of applicants. 

The group-assessment process is a valid predictor of student-teaching success. Other 
instruments and inventories do not have such strong predictive validity. Baskin, Ross, and Smith 
(1996) review the findings of some widely used instruments. Neither the Mirmesota Teacher 
Attitude Inventory, The Teacher Perceiver Interview, nor the Urban Teacher Selection Interview 
were found to predict future teacher success. 

Third, we examined whether the group-assessment interview is a better predictor of 
student-teaching performance than academic criteria such as grades and standardized test scores. 
Not only is the overall-rating score a better predictor of student-teaching performance than 
academic criteria, but for student-teaching evaluation scores given by university supervisors, 
ACT score is negatively associated. In other words, the higher one’s ACT score, the poorer one 
is likely to do in student teaching when evaluated by a university supervisor. This relationship 
holds regardless of how well one scored on the overall group-assessment measure. Grades, at the 
time of admission to the student-teacher program, are not associated with student-teaching 
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evaluation scores. Taken together, our findings show that academic criteria are not positive 
predictors of future student-teaching performance. 

Conclusion 

Shechtman has demonstrated that the group-assessment process is a valid predictor of 
student-teacher success in Israel. Our findings suggest that the group-assessment process is a 
valid process to select candidates into teacher-education programs based on a sample in the 
United States. The group-assessment procedure can assist in the selection process so that socially 
and emotionally mature students can genuinely benefit from teacher-education programs in ways 
that less-mature students would not (Sprinthall, Reiman, & Thies-Sprinthall, 1996). Moreover, 
our results support previous findings that academic criteria that are so pervasively used to select 
teacher-education students have no predictive power to indicate who will be a successful teacher. 
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Table 1 



Standardized Regression Coefficients for Academic-Criteria and Group-Assessment Variables 
Associated with Student-Teaching Evaluation Scores (n=65) 





Cooperating Teacher 


Universitv Supervisor 




Model 


Model 


Model 


Model 


Model Model 




1 


2 


3 


1 


2 3 


Student-Teaching 












Evaluation Score by: 












GPA 


-.06 


-.09 


-.05 


-.14 


-.09 -.12 


ACT 


-.18 


-.18 


-.19 


-.27* 


-.32* -.29* 


Verbal Abilities 




.02 






.08 


Interpersonal Skills 




.21 






.12 


Leadership Qualities 




.18 






.08 


Overall Rating 






.33* 




.34* 


Adjusted 


.12 


.12 


.12 


.11 


.13 .22 



Note. We accept as practically significant the relationships where the unstandardized regression 
coefficient is at least twice the standard error. These relationships are indicated with an asterisk. 
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Table 2 

Student Teaching Evaluation Scores from University Supervisors by ACT Scores and Overall 
Group Assessment Scores 



ACT Score 


HiRh (5-6) 


Overall Group Assessment Score 

Medium ('4') Low (< 31 


Total 






High(>25) 


3.36 (.41)^-'’ 


2.67(1.0) 


2.87 (.60)“’^ 






n=10 


n=4 


n=4 


18 


Medium (22.51-25) 


3.50 (.58)^ 


3.23 (.45) 


3.20 (.61)“ 






n=6 


n=8 


n=5 


19 


Low (19-22.5) 


3.57 (.40)“'’ 


3.37 (.37) 


3.20 (.62)“'^ 






n=15 


n=9 


n=7 


31 


Total 


31 


21 


16 


68 



Note. Because our sample was not randomly selected, it is inappropriate to conduct tests of statistical significance. 
Instead, we examine practical significance using effect size (delta). For the mean differences identified in the table, 
all delta coefficients are >.80. Values of delta greater than .75 are conventionally considered to correspond to 
“medium” effects. 

Delta=(largest mean value - smallest mean value) / sigma, where sigma is the square root of the mean standard 
error. 

®The mean differences between low and high group-assessment scores among respondents is practically 
significantly different across categories of ACT scores, ^e mean differences between low and high ACT scores 
among those earning high overall group-assessment scores are practically significantly different. ‘^The mean 
differences between low and high ACT scores among those earning low group-assessment scores are practically 
significantly different. 
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